Incremental virtual machine metadata extraction

ABSTRACT

A virtual machine container file is analyzed to determine which portion of the virtual machine container file corresponds to a virtual machine file system metadata of the virtual machine container file. One or more differences between a first version of a virtual machine container file and a second version of the virtual machine container file are determined at least in part by traversing a snapshot structure associated with the virtual machine container file. The determined one or more differences that corresponds to the virtual machine file system metadata portion of the virtual machine container file are identified based at least in part on the analysis of the virtual machine container file. The identified one or more differences corresponding to the virtual machine file system metadata portion of the virtual machine file are utilized to identify one or more changes from the content files included in the first version of the virtual machine container file to content files included in the second version of the virtual machine container file.

CROSS REFERENCE TO OTHER APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/705,078, entitled INCREMENTAL VIRTUAL MACHINE METADATA EXTRACTIONfiled Dec. 5, 2019, which is a continuation of U.S. patent applicationSer. No. 16/110,314, now U.S. Pat. No. 10,534,759, entitled INCREMENTALVIRTUAL MACHINE METADATA EXTRACTION filed Aug. 23, 2018, each of whichis incorporated herein by reference for all purposes.

BACKGROUND OF THE INVENTION

A virtual machine that is comprised of a plurality of content files maybe ingested and backed up to a storage system. The storage system maycreate an index of the content files. The virtual machine may be backedup a plurality of times to the storage system and the storage system isconfigured to store the different versions of the virtual machine. Thedifferent versions of the virtual machine may include different versionsof the content files. To determine which files have changed between thevirtual machine versions, conventional systems read the entire contentsof a first and second version of a virtual machine, and determine thedifferences between the virtual machine versions. This is a timeconsuming and resource intensive process because a virtual machine maybe comprised of a large amount of data (e.g., 100 TB).

Other systems read through the metadata associated with a virtualmachine. The metadata associated with a content file of the virtualmachine may include a timestamp. The timestamp may be compared withtimestamps associated with virtual machine versions to determine whenthe content file was modified. The metadata associated with a virtualmachine volume may comprise approximately five percent of the virtualmachine volume. For large virtual machine volumes, going through themetadata to determine which content files have changed based on atimestamp associated with a content file is still a time consuming andresource intensive process.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 is a block diagram illustrating an embodiment of a system forbacking up virtual machines.

FIG. 2A is a block diagram illustrating an embodiment of a tree datastructure.

FIG. 2B is a block diagram illustrating an embodiment of a clonedsnapshot tree.

FIG. 2C is a block diagram illustrating an embodiment of modifying asnapshot tree.

FIG. 2D is a block diagram illustrating an embodiment of a modifiedsnapshot tree.

FIG. 3A is a block diagram illustrating an embodiment of a tree datastructure.

FIG. 3B is a block diagram illustrating an embodiment of adding a filemetadata tree to a tree data structure.

FIG. 3C is a block diagram illustrating an embodiment of modifying afile metadata tree of a tree data structure.

FIG. 3D is a block diagram illustrating an embodiment of a modified filemetadata tree.

FIG. 4 is a flow chart illustrating an embodiment of a process formapping portions of a virtual machine file to a plurality of virtualmachine content files and metadata associated with the plurality ofvirtual machine content files.

FIG. 5 is a flow chart illustrating an embodiment of a process oforganizing file system data of a backup snapshot.

FIG. 6 is a flow chart illustrating an embodiment of a process ofdetermining a modified content file of a virtual machine.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits, and/or processing coresconfigured to process data, such as computer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

A technique to identify one or more virtual machine content files thathave changed or have been added since a previous virtual machine backupis disclosed. The disclosed technique reduces the amount of time andresources needed to identify the one or more virtual machine contentfiles.

A primary system is comprised of file system data. The file system dataincludes a plurality of files and metadata associated with the pluralityof files. The primary system may host one or more virtual machines. Avirtual machine may be stored as one or more container files (e.g.,virtual machine image file, virtual machine disk file, etc.) of theplurality of files of the file system data. The virtual machinecontainer file includes a plurality of virtual machine content files ofthe virtual machine and metadata associated with the plurality ofvirtual machine content files, i.e., virtual machine file systemmetadata. The primary system may perform a backup snapshot of the filesystem data including the one or more virtual machine container filesaccording to a backup policy and send the backup snapshot to a secondarystorage system. A backup snapshot represents the state of the primarysystem at a particular point in time (e.g., the state of the file systemdata). The backup snapshot policy may require a full backup snapshot oran incremental backup snapshot to be performed. A full backup snapshotincludes the entire state of the primary system at a particular point intime. An incremental backup snapshot includes the state of the primarysystem that has changed since a last backup snapshot.

A secondary storage system may ingest and store the backup snapshotacross a plurality of storage nodes of the secondary storage system. Afile system manager of the secondary storage system may organize thefile system data of the backup snapshot using a tree data structure. Anexample of the tree data structure is a snapshot tree (e.g., CohesitySnaptree), which may be based on a B+ tree structure (or other type oftree structure in other embodiments). The tree data structure provides aview of the file system data corresponding to a backup snapshot. Theview of the file system data corresponding to the backup snapshot iscomprised of a snapshot tree and a plurality of file metadata trees(e.g., Blob structures). A file metadata tree may correspond to one ofthe files included in the backup snapshot. The file metadata tree is asnapshot structure that stores the metadata associated with the file.For example, a file metadata tree may correspond to a virtual machinecontainer file (e.g., virtual machine image file, virtual machine diskfile, etc.). Thus, the file metadata tree may store the metadataassociated with a virtual machine container file. Regardless if the viewof the file system data corresponds to a full backup snapshot or anincremental backup snapshot, the view of the file system datacorresponding to the backup snapshot provides a fully hydrated backupsnapshot that provides a complete view of the primary system at a momentin time corresponding to when the backup snapshot was performed. Theview of file system data may allow any content file that was stored onthe primary system at the time the corresponding backup snapshot wasperformed, to be retrieved, restored, or replicated. The view of filesystem data may also allow any content file that was included in avirtual machine container file and was stored on the primary system atthe time the corresponding backup snapshot was performed, to beretrieved, restored, or replicated.

A snapshot tree includes a root node, one or more levels of one or moreintermediate nodes associated with the root node, and one or more leafnodes associated with an intermediate node of the lowest intermediatelevel. The root node of a snapshot tree includes one or more pointers toone or more intermediate nodes. The root node corresponds to aparticular backup snapshot of file system data. Each intermediate nodeincludes one or more pointers to other nodes (e.g., a lower intermediatenode or a leaf node).

Metadata associated with a file that is less than or equal to a limitsize (e.g., 256 kB) may be stored in a leaf node of the snapshot tree.For example, a leaf node may store an inode. Metadata associated with afile that is greater than or equal to the limit size has an associatedfile metadata tree (e.g., Blob structure). The file metadata tree is asnapshot structure and is configured to store the metadata associatedwith a file. The file may correspond to a virtual machine containerfile. Thus, a file metadata tree may be used to represent an entirevirtual machine. The file metadata tree is stored in storage separatelyfrom the file. The file metadata tree includes a root node, one or morelevels of one or more intermediate nodes associated with the root node,and one or more leaf nodes associated with an intermediate node of thelowest intermediate level. A file metadata tree is similar to a snapshottree, but a leaf node of a file metadata tree includes an identifier ofa data brick storing one or more data chunks of the file or a pointer tothe data brick storing one or more data chunks of the file. For example,a leaf node of a file metadata tree may include a pointer to or anidentifier of a data brick storing one or more data chunks of a virtualmachine container file. The location of the data brick may be identifiedusing a table stored in a metadata store that matches brick numbers to aphysical storage location or the location of the data brick may beidentified based on the pointer to the data brick.

A virtual machine container file is comprised of a plurality of virtualmachine content files and metadata associated with the plurality ofvirtual machine content files. The virtual machine container file may beanalyzed to determine which portions of the virtual machine containerfile correspond to the plurality of virtual machine content files andwhich portions of the virtual machine container file correspond to themetadata associated with the plurality of virtual machine content files.The portions of the virtual machine container file corresponding to themetadata associated with the plurality of virtual machine content filesmay be further analyzed to determine which portions of the virtualmachine container file corresponding to the metadata associated with theplurality of virtual machine content files correspond to which virtualmachine content file.

For example, the virtual machine container file may be analyzed todetermine a location of a file table. The virtual machine container filemay store a file table that stores metadata associated with theplurality of virtual machine content files. A first entry of the filetable may correspond to the metadata associated with a first virtualmachine content file, a second entry of the file table may correspond tothe metadata associated with a second virtual machine content file, andan nth entry of the file table may correspond to the metadata associatedwith an nth virtual machine content file. Each entry has an associatedfile offset range within the virtual machine container file. Forexample, a virtual machine container file may be 100 TB and store aplurality of files. The virtual machine container file may store a bootsector in a file offset range of 0-1 kB region of the virtual machinecontainer file and the file table in a 1 kB-100 MB region of the virtualmachine container file. The first entry of the file table may be storedin the file offset range of 1 kB-1.1 kB, the second entry of the filetable may be stored in the file offset range of 1.1 kB-1.2 kB, and annth entry of the file table may be stored in the file offset range of99.9 MB-100 MB.

A snapshot tree may be traversed to determine one or more nodes notshared by two virtual machine versions. A file metadata tree maycorrespond to a version of a virtual machine container file. Thesnapshot tree may include a first leaf node that includes a pointer to afile metadata tree corresponding to the first version of the virtualmachine container file and a second leaf node that includes a pointer toa file metadata tree corresponding to the second version of the virtualmachine container file. The snapshot tree may be traversed from a rootnode of the snapshot tree to the first leaf node and the second leafnode. The file metadata tree corresponding to the first version of thevirtual machine container file and the file metadata tree correspondingto the second version of the virtual machine container file may betraversed to determine one or more leaf nodes that are not shared by thefile metadata trees. In some embodiments, the file metadata treecorresponding to the second version of the virtual machine containerfile is traversed without traversing the file metadata treecorresponding to the first version of the virtual machine container fileto determine one or more leaf nodes that are not shared by the filemetadata trees. The nodes that are not shared by the two versions may bedetermined based on a view identifier associated with a node. Forexample, a node that has a view identifier associated with the secondversion of the virtual machine container file is not included in thefirst version of the virtual machine container file. A leaf node of afile metadata tree may include an identifier of or a pointer to a brickstoring one or more data chunks associated with the virtual machinecontainer file. The data brick storing one or more data chunksassociated with the virtual machine container file may correspond to avirtual machine content file or metadata associated with a virtualmachine content file. The data brick corresponds to a particular fileoffset within the virtual machine container file.

It is determined whether the file offset of the data brick correspondsto a portion of the virtual machine container file that stores a virtualmachine content file or a portion of the virtual machine container filethat stores metadata associated with the virtual machine content file.In the event the data brick corresponds to a portion of the virtualmachine container file that stores the virtual machine content file, thedata brick is ignored and the next data brick is examined. In the eventthe data brick corresponds to a portion of the virtual machine containerfile that stores metadata associated with the virtual machine contentfile, the file offset of the data brick is compared to the file offsetsincluded in the file table. The file offset may be used to determinewhich file has changed between virtual machine container file versions.For example, a data brick with a file offset range of 1 kB-1.1 kBindicates that the metadata associated with a first virtual machinecontent file has been modified. As a result, the first virtual machinecontent file may be determined to have been modified. A data brick witha file offset range of 1.1 kB-1.2 kB indicates that the metadataassociated with a second virtual machine content file has been modified.As a result, the second virtual machine content file may be determinedto have been modified. A data brick with a file offset range of 99.9MB-100 MB indicates that the metadata associated with an nth virtualmachine content file has been modified. As a result, the nth virtualmachine content file may be determined to have been modified.

One or more virtual machine content files that have changed since aprevious virtual machine backup may be quickly identified byintersecting the data bricks identified by traversing the snapshot treewith the portion of a master file table corresponding to modified filesbecause the files in the master file table are small (e.g., 1 kB). Theamount of time needed to read a file in the master file table pales incomparison to the amount of time needed to read all of the virtualmachine metadata. The amount of time needed to read a subset of themaster file table is proportional to the number of virtual machinecontent files that have changed since a last backup. For example, a 100TB virtual machine container file may have 100 GB of metadata. Eachvirtual machine content file may have a corresponding metadata file inthe master file table that is 1 kB in size. Traversing the snapshottrees may identify 10 files have changed since a last backup. Thestorage system may read 10 kB in data (10 files, each metadata file is 1kB) to determine the one or more virtual machine content files that havechanged since a pervious virtual machine backup instead of reading the100 GB of metadata.

The size of metadata associated with a virtual machine content file ismuch smaller than the size of a virtual machine content file. When avirtual machine content file is modified, the amount of metadataassociated with the virtual machine content file that has changed ismuch smaller than the amount of data associated with the virtual machinecontent file that has changed. Examining the metadata associated withthe virtual machine content file to determine if the virtual machinecontent file has changed is faster than examining the data associatedwith the virtual machine content file to determine if the virtualmachine content file has changed because large portions of data notshared by two virtual machine container files may correspond to a singlevirtual machine content file. Examining each aspect of the large portionof data is duplicative because each aspect indicates the single virtualmachine content file has been modified. Whereas, a single portion ofmetadata associated with the single virtual machine container file mayhave changed and the single portion of metadata indicates that thesingle virtual machine content file has changed. Examining the metadataassociated with a virtual machine content file reduces the amount oftime and resources to determine whether the virtual machine content filehas changed or has been added since a previous virtual machine backupbecause the tree data structure enables the portions of metadataassociated with the virtual machine content file that have changed to bequickly identified.

The secondary storage system may manage a map that associates a fileoffset range of metadata associated with a virtual machine content filewith its corresponding virtual machine content file. A leaf node of afile metadata tree corresponding to a virtual machine container file mayindicate a brick storing one or more data chunks of data associated withthe virtual machine container file. The brick has a corresponding fileoffset within the virtual machine container file and may be used todetermine that the brick corresponds to metadata associated with avirtual machine content file. The map may be examined and the fileoffset corresponding to the brick may compared to the file offset rangesof metadata associated with the plurality of virtual machine contentfiles. In the event the file offset corresponding to the brick isincluded in a file offset range associated with metadata associated witha virtual machine content file, the virtual machine content filecorresponding to the file offset range may be determined to have changedor added between virtual machine versions. For example, a data brickwith a file offset of 1.1 kB-1.2 kB corresponds to metadata associatedwith the first virtual machine content file and indicates that the firstvirtual machine content file has been modified, a data brick with a fileoffset of 1.1 kB-1.2 kB corresponds to metadata associated with thesecond virtual machine content file and indicates that the secondvirtual machine content file has been modified, and a data brick with afile offset of 99.9 MB-100 MB corresponds to metadata associated withthe nth virtual machine content file and indicates that the nth virtualmachine content file has been modified.

In other embodiments, for portions of the virtual machine container filethat correspond to metadata associated with a virtual machine contentfile, the metadata associated with a virtual machine content file isread. The metadata may store filename of a virtual machine content fileand a timestamp that indicates that the virtual machine content filewith which the metadata is associated, has changed. For example, themetadata may store a timestamp that indicates the virtual machinecontent file was modified after a last backup snapshot.

Determining which virtual machine content files are not shared betweenvirtual machine versions using the techniques disclosed herein hasseveral advantages. First, an index may be created that lists the one ormore virtual machine content files associated with a virtual machineversion. The amount of time needed to create the index is reducedbecause the one or more virtual machine content files that have beenmodified or added since a previous virtual machine version may bequickly identified using the techniques disclosed herein. The indexassociated with a previous version of the virtual machine may be quicklyupdated to include the one or more identified virtual machine contentfiles. Second, a version of a virtual machine content file includedwithin a virtual machine version may be determined. This may enable auser to recover a particular version of a virtual machine content file.Third, a virus scan of a virtual machine may be performed a lot faster.Typically, a virus scanner may scan a first version of a virtual machine(e.g., the entire virtual machine container file) to determine whetherthere is a problem with any of the virtual machine content filesincluded in the first version of the virtual machine container file. Aconventional system may also scan a second version of the virtualmachine to determine whether there is a problem with any of the virtualmachine content files included in the second version virtual machinecontainer file. Instead of scanning the entire contents of the secondversion of the virtual machine, the one or more virtual machine contentfiles that have been modified or added since the first version of thevirtual machine may be scanned. The one or more virtual machine contentfiles may be quickly identified using the techniques disclosed herein.Subsequently, a virus scanner may be applied to the portions of thevirtual machine container file corresponding to the one or moreidentified virtual machine content files. This reduces the amount oftime to perform a virus scan of the virtual machine container file.Fourth, the virtual machine container file may be analyzed to determinehow much data has changed between virtual machine versions and whichportions of the virtual machine container file have changed. This mayallow a user of the virtual machine container file to determine whichportions of the virtual machine are frequently used and/or critical tothe operation of the virtual machine.

FIG. 1 is a block diagram illustrating an embodiment of a system forbacking up virtual machines. In the example shown, system 100 includes aprimary system 102 and a secondary storage system 112.

Primary system 102 is a computing system that stores file system data.The file system data may be stored across one or more object(s), virtualmachine(s), physical entity/entities, file system(s), array backup(s),and/or volume(s) of the primary system 102. Primary system 102 may becomprised of one or more servers, one or more computing devices, one ormore storage devices, and/or a combination thereof.

Primary system 102 may include one or more virtual machines 104. Avirtual machine may be stored as one or more container files (e.g.,virtual machine image file, virtual machine disk file, etc.). Thevirtual machine container file includes a plurality of virtual machinecontent files of the virtual machine and metadata associated with theplurality of virtual machine content files.

Primary system 102 may be configured to backup file system data tosecondary storage system 112 according to one or more backup policies.The file system data includes the one or more virtual machine containerfiles corresponding to the one or more virtual machines 104. In someembodiments, a backup policy indicates that file system data is to bebacked up on a periodic basis (e.g., hourly, daily, weekly, monthly,etc.). In other embodiments, a backup policy indicates that file systemdata is to be backed up when a threshold size of data has changed. Inother embodiments, a backup policy indicates that file system data is tobe backed up upon a command from a user associated with primary system102. The backup policy may indicate when a full backup snapshot is to beperformed and when an incremental backup snapshot is to be performed.For example, the backup policy may indicate that a full backup snapshotis to be performed according to a first schedule (e.g., weekly, monthly,etc.) and an incremental backup snapshot is to be performed according toa second schedule (e.g., hourly, daily, weekly, etc.) The backup policymay indicate that a full backup snapshot is to be performed after athreshold number of incremental backup snapshots have been performed.

Secondary storage system 112 is a storage system configured to storefile system data received from primary storage system 102. Secondarystorage system 112 may protect a large volume of applications whilesupporting tight business requirements (recovery time objective (RTO)and recovery point objective (RPO)). Secondary storage system 112 mayunify end-to-end protection infrastructure—including target storage,provide backup, replication of data, disaster recovery, and/or cloudtiering. Secondary storage system 112 may provide scale-out, globallydeduped, highly available storage to consolidate all secondary data,including backups, files, and test/dev copies. Secondary storage system112 simplifies backup infrastructure and eliminates the need to runseparate backup software, proxies, media servers, and archival.Secondary storage system 112 may be fully integrated with a virtualmachine (VM) centralized management tool, such as vCenter, and anapplications programming interface (API) for data protection. Secondarystorage system 112 may reduce the amount of time to perform RPOs andsupport instantaneous RTOs by creating a clone of a backup VM andrunning the VM directly from secondary storage system 112. Secondarystorage system 112 may integrate natively with one or more cloudservers. Secondary storage system 112 may replicate data to a one ormore cloud clusters to minimize potential data loss by replicating dataas soon as a backup is completed. This allows data in the cloud to beused for disaster recovery, application migration, test/dev, oranalytics.

Secondary storage system 112 may be comprised of one or more storagenodes 111, 113, 117. The one or more storage nodes may be one or moresolid state drives, one or more hard disk drives, or a combinationthereof. The file system data included in a backup snapshot may bestored in one or more of the storage nodes 111, 113, 117. In oneembodiment, secondary storage system 112 is comprised of one solid statedrive and three hard disk drives.

Secondary storage system 112 may include a file system manager 115. Filesystem manager 115 is configured to organize the file system data in atree data structure. An example of the tree data structure is a snapshottree (e.g., Cohesity Snaptree), which may be based on a B+ treestructure (or other type of tree structure in other embodiments). Thetree data structure provides a view of the file system datacorresponding to a backup snapshot. The view of the file system datacorresponding to the backup snapshot is comprised of a snapshot tree anda plurality of file metadata trees (e.g., blob structures). A filemetadata tree may correspond to one of the files included in the backupsnapshot. The file metadata tree is a snapshot structure that stores themetadata associated with the file. For example, a file metadata tree maycorrespond to a virtual machine container file (e.g., virtual machineimage file, virtual machine disk file, etc.). Thus, the file metadatatree may store virtual machine file system metadata. The tree datastructure may include one or more leaf nodes that store a data key-valuepair. A user may request a particular value by providing a particulardata key to file system manager 115, which traverses a view of a backupsnapshot to find the value associated with the particular data key. Auser may request a set of content files within a particular range ofdata keys of a snapshot. File system manager 115 may be configured togenerate a view of file system data based on a backup snapshot receivedfrom primary system 102. File system manager 115 may be configured toperform one or more modifications, as disclosed herein, to a snapshottree. The snapshot trees and file metadata trees may be stored inmetadata store 114. The metadata store 114 may store the view of filesystem data corresponding to a backup snapshot. The metadata store mayalso store metadata associated with content files that are smaller thana limit size.

The tree data structure may be used to capture different versions ofbackup snapshots. The tree data structure allows a chain of snapshottrees corresponding to different versions of backup snapshots (i.e.,different snapshot tree versions) to be linked together by allowing anode of a later version of a snapshot tree to reference a node of aprevious version of a snapshot tree (e.g., a “snapshot tree forest”).For example, a root node or an intermediate node of the second snapshottree corresponding to the second backup snapshot may reference anintermediate node or leaf node of the first snapshot tree correspondingto a first backup snapshot. The snapshot tree provides a view of thefile system data corresponding to a backup snapshot.

A snapshot tree includes a root node, one or more levels of one or moreintermediate nodes associated with the root node, and one or more leafnodes associated with an intermediate node of the lowest intermediatelevel. The root node of a snapshot tree includes one or more pointers toone or more intermediate nodes. Each intermediate node includes one ormore pointers to other nodes (e.g., a lower intermediate node or a leafnode). A leaf node may store file system metadata, an identifier of adata brick, a pointer to a file metadata tree (e.g., Blob structure), ora pointer to a data chunk stored on the secondary storage system. A leafnode may correspond to a data brick. The data brick may have acorresponding brick number.

Metadata associated with a file that is smaller than or equal to a limitsize (e.g., 256 kB) may be stored in a leaf node of the snapshot tree.For example, a leaf node may store an inode. Metadata associated with afile that is larger than the limit size may be stored across the one ormore storage nodes 111, 113, 117. A file metadata tree may be generatedfor the metadata associated with a file that is larger than the limitsize. The file metadata tree is a snapshot structure and is configuredto store the metadata associated with a file. The file may correspond toa virtual machine container file (e.g., virtual machine image file,virtual machine disk file, etc.). Thus, a file metadata tree may be usedto represent an entire virtual machine.

The file metadata tree includes a root node, one or more levels of oneor more intermediate nodes associated with the root node, and one ormore leaf nodes associated with an intermediate node of the lowestintermediate level. A file metadata tree is similar to a snapshot tree,but a leaf node of a file metadata tree includes an identifier of a databrick storing one or more data chunks of the file or a pointer to thedata brick storing one or more data chunks of the file. For example, aleaf node of a file metadata tree may include a pointer to or anidentifier of a data brick storing one or more data chunks of a virtualmachine container file. The location of the data brick may be identifiedusing a table stored in a metadata store that matches brick numbers to aphysical storage location or the location of the data brick may beidentified based on the pointer to the data brick. The data of a file,such as a virtual machine container file, may be divided into aplurality of bricks. A leaf node of a file metadata tree may correspondto one of the plurality of bricks. A leaf node of the file metadata treemay include a pointer to a storage location for the brick. In someembodiments, the size of a brick is 256 kB.

A virtual machine container file is comprised of a plurality of virtualmachine content files and metadata associated with the plurality ofvirtual machine content files. File system manager 115 may analyze thevirtual machine container file to determine which portions of thevirtual machine container file correspond to the plurality of virtualmachine content files and which portions of the virtual machinecontainer file correspond to the metadata associated with the pluralityof virtual machine content files. File system manager 115 may furtheranalyze the portions of the virtual machine container file correspondingto the metadata associated with the plurality of virtual machine contentfiles to determine which portions of the virtual machine container filecorresponding to the metadata associated with the plurality of virtualmachine content files correspond to which virtual machine content file.

For example, file system manager 115 may analyze the virtual machinecontainer file to determine a location of a file table. The virtualmachine container file may store a file table that stores metadataassociated with the plurality of virtual machine content files. A firstentry of the file table may correspond to the metadata associated with afirst virtual machine content file, a second entry of the file table maycorrespond to the metadata associated with a second virtual machinecontent file, and an nth entry of the file table may correspond to themetadata associated with an nth virtual machine content file. Each entryhas an associated file offset range within the virtual machine containerfile. For example, a virtual machine container file may be 100 TB andstore a plurality of files. The virtual machine container file may storea boot sector in a file offset range of 0-1 kB region of the virtualmachine container file and the file table in a 1 kB-100 MB region of thevirtual machine container file. The first entry of the file table may bestored in the file offset range of 1 kB-1.1 kB, the second entry of thefile table may be stored in the file offset range of 1.1 kB-1.2 kB, andan nth entry of the file table may be stored in the file offset range of99.9 MB-100 MB. File system manager 115 may generate a map thatassociates portions of the file table with their corresponding virtualmachine content file.

File system manager 115 may traverse a snapshot tree to determine one ormore nodes not shared by two virtual machine versions. A file metadatatree may correspond to a version of a virtual machine container file.The snapshot tree may include a first leaf node that includes a pointerto a file metadata tree corresponding to the first version of thevirtual machine container file and a second leaf node that includes apointer to a file metadata tree corresponding to the second version ofthe virtual machine container file. File system manager 115 may traversethe snapshot tree from a root node of the snapshot tree to the firstleaf node and the second leaf node. File system manager 115 may traversethe file metadata tree corresponding to the first version of the virtualmachine container file and the file metadata tree corresponding to thesecond version of the virtual machine container file to determine one ormore leaf nodes that are not shared by the file metadata trees. In someembodiments, file system manager 115 traverses the file metadata treecorresponding to the second version of the virtual machine containerfile without traversing the file metadata tree corresponding to thefirst version of the virtual machine container file to determine one ormore leaf nodes that are not shared by the file metadata trees. Thenodes that are not shared by the two versions may be determined based ona view identifier associated with a node. For example, a node that has aview identifier associated with the second version of the virtualmachine container file is not included in the first version of thevirtual machine container file. A leaf node of a file metadata tree mayinclude an identifier of or a pointer to a brick storing one or moredata chunks associated with the virtual machine container file. The databrick storing one or more data chunks associated with the virtualmachine container file may correspond to a virtual machine content fileor metadata associated with a virtual machine content file. The databrick corresponds to a particular file offset within the virtual machinecontainer file.

File system manager 115 may determine whether the file offset of thedata brick corresponds to a portion of the virtual machine containerfile that stores a virtual machine content file or a portion of thevirtual machine container file that stores metadata associated with thevirtual machine content file. In the event the data brick corresponds toa portion of the virtual machine container file that stores the virtualmachine content file, file system manager 115 may ignore the data brickand examine the next data brick. In the event the data brick correspondsto a portion of the virtual machine container file that stores metadataassociated with the virtual machine content file, file system manager115 may compare the file offset of the data brick to the file offsetsincluded in the file table. The file offset may be used to determinewhich file has changed between virtual machine container file versions.For example, a data brick with a file offset range of 1 kB-1.1 kB storesmetadata associated with a first virtual machine content file andindicates that the metadata associated with the first virtual machinecontent file has been modified. A data brick with a file offset range of1.1 kB-1.2 kB stores metadata associated with a second virtual machinecontent file and indicates that the second virtual machine content filehas been modified. A data brick with a file offset range of 99.9 MB-100MB stores metadata associated with an nth virtual machine content fileand indicates that the nth virtual machine content file has beenmodified.

File system manager 115 may manage a map that associates file offsetranges with virtual machine content files. The map may be stored inmetadata store 114. A leaf node of a file metadata tree corresponding toa virtual machine container file may indicate a brick storing one ormore data chunks of data associated with the virtual machine containerfile. The brick has a corresponding file offset and may be used by filesystem manager 115 to determine that the brick corresponds to metadataassociated with a virtual machine content file. File system manager maycompare the file offset corresponding to the brick to the file offsetrange associated with metadata associated with the plurality of virtualmachine content files. In the event the file offset corresponding to thebrick is included in a file offset range associated with metadataassociated with a virtual machine content file, file system manager 115may determine that the virtual machine content file corresponding to thefile offset range has changed or been added between virtual machineversions. For example, a data brick with a file offset of 1.1 kB-1.2 kBindicates that a first virtual machine content file has been modified, adata brick with a file offset of 1.1 kB-1.2 kB indicates that a secondvirtual machine content file has been modified, and a data brick with afile offset of 99.9 MB-100 MB indicates that an nth virtual machinecontent file has been modified.

In other embodiments, for portions of the virtual machine container filethat correspond to metadata associated with a virtual machine contentfile, file system manager 115 may read the metadata associated with avirtual machine content file. The metadata may store filename of avirtual machine content file and a timestamp that indicates that thevirtual machine content file with which the metadata is associated, haschanged. For example, the metadata may store a timestamp that indicatesthe virtual machine content file was modified after a last backupsnapshot. The metadata associated with a virtual machine content filemay be read and the virtual machine content file with which the metadatais associated, is determined to have changed.

FIG. 2A is a block diagram illustrating an embodiment of a tree datastructure. A tree data structure may be used to represent the filesystem data that is stored on a secondary storage system, such assecondary storage system 112. The file system data may include metadatafor a distributed file system and may include information, such as chunkidentifier, chunk offset, file size, directory structure, filepermissions, physical storage locations of the files, etc. A file systemmanager, such as file system manager 115, may generate tree datastructure 200.

Tree data structure 200 is comprised of a snapshot tree that includes aroot node 202, intermediate nodes 212, 214, and leaf nodes 222, 224,226, 228, and 230. Although tree data structure 200 includes oneintermediate level between root node 202 and leaf nodes 222, 224, 226,228, 230, any number of intermediate levels may be implemented. Treedata structure 200 may correspond to a backup snapshot of file systemdata at a particular point in time t, for example at time to. The backupsnapshot may be received from a primary system, such as primary system102. The snapshot tree in conjunction with a plurality of file metadatatrees may provide a complete view of the primary system associated withthe backup snapshot for the particular point in time.

A root node is the starting point of a snapshot tree and may includepointers to one or more other nodes. An intermediate node is a node towhich another node points (e.g., root node, other intermediate node) andincludes one or more pointers to one or more other nodes. A leaf node isa node at the bottom of a snapshot tree. Each node of the tree structureincludes a view identifier of a view with which the node is associated(e.g., TreeID).

A leaf node may be configured to store key-value pairs of file systemdata. A data key k is a lookup value by which a particular leaf node maybe accessed. For example, “1” is a data key that may be used to lookup“DATA1” of leaf node 222. The data key k may correspond to a bricknumber of a data brick. A data brick may be comprised of one or moredata blocks. In some embodiments, the leaf node is configured to storefile system metadata (e.g., chunk identifier (e.g., hash value, SHA-1,etc.), file size, directory structure, file permissions, physicalstorage locations of the files, etc.). A leaf node may store a data keyk and a pointer to a location that stores the value associated with thedata key.

In other embodiments, a leaf node is configured to store the actual datawhen the metadata associated with a file is less than or equal to alimit size. For example, metadata associated with a file that is lessthan or equal to 256 kB may reside in the leaf node of a snapshot tree.In some embodiments, a leaf node includes a pointer to a file metadatatree (e.g., blob structure) when the size of metadata associated with afile is larger than the limit size. For example, a leaf node may includea pointer to a file metadata tree corresponding to a virtual machinecontainer file.

A root node or an intermediate node may include one or more node keys.The node key may be an integer value or a non-integer value. Each nodekey indicates a division between the branches of the node and indicateshow to traverse the tree structure to find a leaf node, i.e., whichpointer to follow. For example, root node 202 may include a node key of“3.” A data key k of a key-value pair that is less than or equal to thenode key is associated with a first branch of the node and a data key kof a key-value pair that is greater than the node key is associated witha second branch of the node. In the above example, to find a leaf nodestoring a value associated with a data key of “1,” “2,” or “3,” thefirst branch of root node 202 would be traversed to intermediate node212 because the data keys of “1,” “2”, and “3” are less than or equal tothe node key “3.” To find a leaf node storing a value associated with adata key of “4” or “5,” the second branch of root node 202 would betraversed to intermediate node 214 because data keys “4” and “5” aregreater than the node key of “3.”

In some embodiments, a hash function may determine which branch of anode with which the non-numerical key is associated. For example, a hashfunction may determine that a first bucket is associated with a firstbranch of a node and a second bucket is associated with a second branchof the node.

A data key k of a key-value pair is not limited to a numerical value. Insome embodiments, non-numerical data keys may be used for a datakey-value pair (e.g., “name,” “age”, etc.) and a numerical number may beassociated with the non-numerical data key. For example, a data key of“name” may correspond to a numerical key of “3.” Data keys thatalphabetically come before the word “name” or is the word “name” may befound following a left branch associated with a node. Data keys thatalphabetically come after the word “name” may be found by following aright branch associated with the node. In some embodiments, a hashfunction may be associated with the non-numerical data key. The hashfunction may determine which branch of a node with which thenon-numerical data key is associated.

In the example shown, root node 202 includes a pointer to intermediatenode 212 and a pointer to intermediate node 214. Root node 202 includesa NodeID of “R1” and a TreeD of “1.” The NodeID identifies the name ofthe node. The TreeID identifies the view with which the node isassociated. When a change is made to data stored in a leaf node asdescribed with respect to FIGS. 2B, 2C, and 2D, the TreeID is used todetermine whether a copy of a node is to be made.

Root node 202 includes a node key that divides a set of pointers intotwo different subsets. Leaf nodes (e.g., “1-3”) with a data key k thatis less than or equal to the node key are associated with a first branchand leaf nodes (e.g., “4-5”) with a data key k that is greater than thenode key are associated with a second branch. Leaf nodes with a data keyof “1,” “2,” or “3” may be found by traversing tree data structure 200from root node 202 to intermediate node 212 because the data keys have avalue that is less than or equal to the node key. Leaf nodes with a datakey of “4” or “5” may be found by traversing tree data structure 200from root node 202 to intermediate node 214 because the data keys have avalue that is greater than the node key.

Root node 202 includes a first set of pointers. The first set ofpointers associated with a data key less than the node key (e.g., “1”,“2,” or “3”) indicates that traversing tree data structure 200 from rootnode 202 to intermediate node 212 will lead to a leaf node with a datakey of “1,” “2,” or “3.” Intermediate node 214 includes a second set ofpointers. The second set of pointers associated with a data key greaterthan the node key indicates that traversing tree data structure 200 fromroot node 202 to intermediate node 214 will lead to a leaf node with adata key of “4” or “5.”

Intermediate node 212 includes a pointer to leaf node 222, a pointer toleaf node 224, and a pointer to leaf node 226. Intermediate node 212includes a NodeID of “I1” and a TreeID of “1.” Intermediate node 212includes a first node key of “1” and a second node key of “2.” The datakey k for leaf node 222 is a value that is less than or equal to thefirst node key. The data key k for leaf node 224 is a value that isgreater than the first node key and less than or equal to the secondnode key. The data key k for leaf node 226 is a value that is greaterthan the second node key. The pointer to leaf node 222 indicates thattraversing tree data structure 200 from intermediate node 212 to leafnode 222 will lead to the node with a data key of “1.” The pointer toleaf node 224 indicates that traversing tree data structure 200 fromintermediate node 212 to leaf node 224 will lead to the node with a datakey of “2.” The pointer to leaf node 226 indicates that traversing treedata structure 200 from intermediate node 212 to leaf node 226 will leadto the node with a data key of “3.”

Intermediate node 214 includes a pointer to leaf node 228 and a pointerto leaf node 230. Intermediate node 212 includes a NodeID of “I2” and aTreeID of “1.” Intermediate node 214 includes a node key of “4.” Thedata key k for leaf node 228 is a value that is less than or equal tothe node key. The data key k for leaf node 230 is a value that isgreater than the node key. The pointer to leaf node 228 indicates thattraversing tree data structure 200 from intermediate node 214 to leafnode 228 will lead to the node with a data key of “4.” The pointer toleaf node 230 indicates that traversing tree data structure 200 fromintermediate node 214 to leaf node 230 will lead the node with a datakey of “5.”

Leaf node 222 includes a data key-value pair of “1: DATA1.” Leaf node222 includes NodeID of “L1” and a TreeID of “1.” To view the valueassociated with a data key of “1,” tree data structure 200 is traversedfrom root node 202 to intermediate node 212 to leaf node 222. In someembodiments, leaf node 222 is configured to store metadata associatedwith a file. In other embodiments, leaf node 222 is configured to storea pointer to a file metadata tree (e.g., blob structure). For example,the file metadata tree may correspond to a virtual machine containerfile.

Leaf node 224 includes a data key-value pair of “2: DATA2.” Leaf node224 includes NodeID of “L2” and a TreeID of “1.” To view the valueassociated with a data key of “2,” tree data structure 200 is traversedfrom root node 202 to intermediate node 212 to leaf node 224. In someembodiments, leaf node 224 is configured to store metadata associatedwith a file. In other embodiments, leaf node 224 is configured to storea pointer to a file metadata tree (e.g., blob structure). For example,the file metadata tree may correspond to a virtual machine containerfile.

Leaf node 226 includes a data key-value pair of “3: DATA3.” Leaf node226 includes NodeID of “L3” and a TreeID of “1.” To view the valueassociated with a data key of “3,” tree data structure 200 is traversedfrom root node 202 to intermediate node 212 to leaf node 226. In someembodiments, leaf node 226 is configured to store metadata associatedwith a file. In other embodiments, leaf node 226 is configured to storea pointer to a file metadata tree (e.g., blob structure). For example,the file metadata tree may correspond to a virtual machine containerfile.

Leaf node 228 includes a data key-value pair of “4: DATA4.” Leaf node228 includes NodeID of “L4” and a TreeID of “1.” To view the valueassociated with a data key of “4,” tree data structure 200 is traversedfrom root node 202 to intermediate node 214 to leaf node 228. In someembodiments, leaf node 228 is configured to store metadata associatedwith a file. In other embodiments, leaf node 228 is configured to storea pointer to a file metadata tree (e.g., blob structure). For example,the file metadata tree may correspond to a virtual machine containerfile.

Leaf node 230 includes a data key-value pair of “5: DATA5.” Leaf node230 includes NodeID of “L5” and a TreeID of “1.” To view the valueassociated with a data key of “5,” tree data structure 200 is traversedfrom root node 202 to intermediate node 214 to leaf node 230. In someembodiments, leaf node 230 is configured to store metadata associatedwith a file. In other embodiments, leaf node 230 is configured to storea pointer to a file metadata tree (e.g., blob structure). For example,the file metadata tree may correspond to a virtual machine containerfile.

FIG. 2B is a block diagram illustrating an embodiment of a clonedsnapshot tree. A snapshot tree may be cloned when a snapshot tree isadded to a tree data structure. In some embodiments, tree data structure250 may be created by a storage system, such as secondary storage system112. The file system data of a primary system, such as primary system102, may be backed up to a secondary storage system, such as secondarystorage system 112. A subsequent backup snapshot may correspond to afull backup snapshot or an incremental backup snapshot. The manner inwhich the file system data corresponding to the subsequent backupsnapshot is stored in secondary storage system may be represented by atree data structure. The tree data structure corresponding to thesubsequent backup snapshot is created by cloning a snapshot treeassociated with a last backup.

In the example shown, tree data structure 250 includes root nodes 202,204, intermediate nodes 212, 214, and leaf nodes 222, 224, 226, 228, and230. Tree data structure 250 may be a snapshot of file system data at aparticular point in time t+n. The tree data structure can be used tocapture different versions of file system data at different moments intime. The tree data structure may also efficiently locate desiredmetadata by traversing a particular version of a snapshot tree includedin the tree data structure. In some embodiments, the tree data structureallows a chain of backup snapshot versions (i.e., snapshot trees) to belinked together by allowing a node of a later version of a snapshot treeto reference a node of a previous version of a snapshot tree. Forexample, a snapshot tree with root node 204 is linked to a snapshot treewith root node 202. Each time a snapshot is performed, a new root nodemay be created and the new root node includes the same set of pointersincluded in the previous root node, that is the new root node of thesnapshot may be linked to one or more intermediate nodes associated witha previous snapshot. The new root node also includes a different NodeIDand a different TreeID. The TreeID is the view identifier associatedwith a view of the primary system associated with the backup snapshotfor the particular moment in time.

In some embodiments, a root node is associated with a current view ofthe file system data. A current view may still accept one or morechanges to the data. The TreeID of a root node indicates a snapshot withwhich the root node is associated. For example, root node 202 with aTreeID of “1” is associated with a first backup snapshot and root node204 with a TreeID of “2” is associated with a second backup snapshot. Inthe example shown, root node 204 is associated with a current view ofthe file system data.

In other embodiments, a root node is associated with a snapshot view ofthe file system data. A snapshot view may represent a state of the filesystem data at a particular moment in time in the past and is notupdated. In the example shown, root node 202 is associated with asnapshot view of the file system data.

In the example shown, root node 204 is a copy of root node 202. Similarto root node 202, root node 204 includes the same pointers as root node202. Root node 204 includes a first set of pointers to intermediate node212. The first set of pointers associated with a data key k less than orequal to the node key (e.g., “1,” “2,” or “3”) indicates that traversingtree data structure 250 from root node 204 to intermediate node 212 willlead to a leaf node with a data key of “1,” “2,” or “3.” Root node 204includes a second set of pointers to intermediate node 214. The secondset of pointers associated with a data key k greater than the node keyindicates that traversing tree data structure 250 from root node 204 tointermediate node 214 will lead to a leaf node with a data key of “4” or“5.” Root node 204 includes a NodeID of “R2” and a TreeID of “2.” TheNodeID identifies the name of the node. The TreeID identifies the backupsnapshot with which the node is associated.

FIG. 2C is a block diagram illustrating an embodiment of modifying asnapshot tree. In the example shown, tree data structure 255 may bemodified by a file system manager, such as file system manager 115. Asnapshot tree with a root node 204 may be a current view of the filesystem data at time t+n+m, for example, at time t₂. A current viewrepresents a state of the file system data that is up-to-date andcapable of receiving one or more modifications to the snapshot tree thatcorrespond to modifications to the file system data. Because a snapshotrepresents a perspective of the file system data that is “frozen” intime, one or more copies of one or more nodes affected by a change tofile system data, are made.

In the example shown, the value “DATA4” has been modified to be“DATA4′.” In some embodiments, the value of a key value pair has beenmodified. For example, the value of “DATA4” may be a pointer to a filemetadata tree corresponding to a first version of a virtual machine andthe value of “DATA4′” may be a pointer to a file metadata treecorresponding to the second version of the virtual machine. In otherembodiments, the value of the key pair is the data of metadataassociated with a content file that is smaller than or equal to a limitsize. In other embodiments, the value of the key value pair points to adifferent file metadata tree. The different file metadata tree may be amodified version of the file metadata tree that the leaf node previouslypointed.

At t₂, the file system manager starts at root node 204 because that isthe root node associated with snapshot tree at time t₂ (i.e., the rootnode associated with the last backup snapshot). The value “DATA4” isassociated with the data key “4.” The file system manager traversessnapshot tree 255 from root node 204 until it reaches a target node, inthis example, leaf node 228. The file system manager compares the TreeIDat each intermediate node and leaf node with the TreeID of the rootnode. In the event the TreeID of a node matches the TreeID of the rootnode, the file system manager proceeds to the next node. In the eventthe TreeID of a node does not match the TreeID of the root node, ashadow copy of the node with the non-matching TreeID is made. Forexample, to reach a leaf node with a data key of “4,” the file systemmanager begins at root node 204 and proceeds to intermediate node 214.The file system manager compares the TreeID of intermediate node 214with the TreeID of root node 204, determines that the TreeID ofintermediate node 214 does not match the TreeID of root node 204, andcreates a copy of intermediate node 214. The intermediate node copy 216includes the same set of pointers as intermediate node 214, but includesa TreeID of “2” to match the TreeID of root node 204. The file systemmanager updates a pointer of root node 204 to point to intermediate node216 instead of pointing to intermediate node 214. The file systemmanager traverses tree data structure 255 from intermediate node 216 toleaf node 228, determines that the TreeID of leaf node 228 does notmatch the TreeID of root node 204, and creates a copy of leaf node 228.Leaf node copy 232 stores the modified value “DATA4′” and includes thesame TreeID as root node 204. The file system manager updates a pointerof intermediate node 216 to point to leaf node 232 instead of pointingto leaf node 228.

In some embodiments, leaf node 232 stores the value of a key value pairthat has been modified. In other embodiments, leaf node 232 stores themodified data of metadata associated with a file that is smaller than orequal to a limit size. In other embodiments, leaf node 232 stores apointer to a file metadata tree corresponding to a file, such as avirtual machine container file.

FIG. 2D is a block diagram illustrating an embodiment of a modifiedsnapshot tree. Tree data structure 255 shown in FIG. 2D illustrates aresult of the modifications made to a snapshot tree as described withrespect to FIG. 2C.

FIG. 3A is a block diagram illustrating an embodiment of a tree datastructure. In some embodiments, tree data structure 300 may be createdby a storage system, such as secondary storage system 112. In theexample shown, tree data structure 300 corresponds to a file and storesthe metadata associated with the file. For example, tree data structure300 may correspond to a virtual machine container file and may be usedto store virtual machine file system metadata. The metadata associatedwith a file is stored by a storage system as a file separate from thefile with which the metadata is associated, that is, the tree datastructure is stored separately from a file. A leaf node of a snapshottree associated with file system data, such as a leaf node of tree datastructures 200, 250, 255, may include a pointer to a tree data structurecorresponding to a file, such as tree data structure 300. A tree datastructure corresponding to a file (i.e., a “file metadata tree”) is asnapshot tree, but is used to organize the data blocks associated with afile that are stored on the secondary storage system. Tree datastructure 300 may be referred to as a “metadata structure” or a“snapshot structure.”

A tree data structure corresponding to a content file at a particularpoint in time (e.g., a particular version) may be comprised of a rootnode, one or more levels of one or more intermediate nodes, and one ormore leaf nodes. In some embodiments, a tree data structurecorresponding to a content file is comprised of a root node and one ormore leaf nodes without any intermediate nodes. Tree data structure 300may be a snapshot of a content file at a particular point in time t, forexample at time to. A tree data structure associated with file systemdata may include one or more pointers to one or more tree datastructures corresponding to one or more content files.

In the example shown, tree data structure 300 includes a file root node302, file intermediate nodes 312, 314, and file leaf nodes 322, 324,326, 328, 330. Although tree data structure 300 includes oneintermediate level between root node 302 and leaf nodes 322, 324, 326,328, 330, any number of intermediate levels may be implemented. Similarof the snapshot trees described above, each node includes a “NodeID”that identifies the node and a “TreeID” that identifies a snapshot/viewwith which the node is associated.

In the example shown, root node 302 includes a pointer to intermediatenode 312 and a pointer to intermediate node 314. Root node 202 includesa NodeID of “FR1” and a TreeID of “1.” The NodeID identifies the name ofthe node. The TreeID identifies the snapshot/view with which the node isassociated.

In the example shown, intermediate node 312 includes a pointer to leafnode 322, a pointer to leaf node 324, and a pointer to leaf node 326.Intermediate node 312 includes a NodeID of “FI1” and a TreeID of “1.”Intermediate node 312 includes a first node key and a second node key.The data key k for leaf node 322 is a value that is less than or equalto the first node key. The data key for leaf node 324 is a value that isgreater than the first node key and less than or equal to the secondnode key. The data key for leaf node 326 is a value that is greater thanthe second node key. The pointer to leaf node 322 indicates thattraversing tree data structure 300 from intermediate node 312 to leafnode 322 will lead to the node with a data key of “1.” The pointer toleaf node 324 indicates that traversing tree data structure 300 fromintermediate node 312 to leaf node 324 will lead to the node with a datakey of “2.” The pointer to leaf node 326 indicates that traversing treedata structure 300 from intermediate node 312 to leaf node 326 will leadto the node with a data key of “3.”

In the example shown, intermediate node 314 includes a pointer to leafnode 328 and a pointer to leaf node 330. Intermediate node 314 includesa NodeID of “FI2” and a TreeID of “1.” Intermediate node 314 includes anode key. The data key k for leaf node 328 is a value that is less thanor equal to the node key. The data key for leaf node 330 is a value thatis greater than the node key. The pointer to leaf node 328 indicatesthat traversing tree data structure 300 from intermediate node 314 toleaf node 328 will lead to the node with a data key of “4.” The pointerto leaf node 330 indicates that traversing tree data structure 300 fromintermediate node 314 to leaf node 330 will lead the node with a datakey of “5.”

Leaf node 322 includes a data key-value pair of “1: Brick 1.” “Brick 1”is a brick identifier that identifies the data brick storing one or moredata chunks associated with a content file corresponding to tree datastructure 300. “Brick 1” may store one or more data chunks associatedwith a virtual machine content file or one or more data chunks ofmetadata associated with the virtual machine content file. Leaf node 322includes NodeID of “FL1” and a TreeID of “1.” To view the valueassociated with a data key of “1,” tree data structure 300 is traversedfrom root node 302 to intermediate node 312 to leaf node 322.

Leaf node 324 includes a data key-value pair of “2: Brick 2.” “Brick 2”is a brick identifier that identifies the data brick storing one or moredata chunks associated with a content file corresponding to tree datastructure 300. “Brick 2” may store one or more data chunks associatedwith a virtual machine content file or one or more data chunks ofmetadata associated with the virtual machine content file. Leaf node 324includes NodeID of “FL2” and a TreeID of “1.” To view the valueassociated with a data key of “2,” tree data structure 300 is traversedfrom root node 302 to intermediate node 312 to leaf node 324.

Leaf node 326 includes a data key-value pair of “3: Brick 3.” “Brick 3”is a brick identifier that identifies the data brick storing one or moredata chunks associated with a content file corresponding to tree datastructure 300. “Brick 3” may store one or more data chunks associatedwith a virtual machine content file or one or more data chunks ofmetadata associated with the virtual machine content file. Leaf node 326includes NodeID of “FL3” and a TreeID of “1.” To view the valueassociated with a data key of “3,” tree data structure 300 is traversedfrom root node 302 to intermediate node 312 to leaf node 326.

Leaf node 328 includes a data key-value pair of “4: Brick 4.” “Brick 4”is a brick identifier that identifies the data brick storing one or moredata chunks associated with a content file corresponding to tree datastructure 300. “Brick 4” may store one or more data chunks associatedwith a virtual machine content file or one or more data chunks ofmetadata associated with the virtual machine content file. Leaf node 328includes NodeID of “FL4” and a TreeID of “1.” To view the valueassociated with a data key of “4,” tree data structure 300 is traversedfrom root node 302 to intermediate node 314 to leaf node 328.

Leaf node 330 includes a data key-value pair of “5: Brick 5.” “Brick 5”is a brick identifier that identifies the data brick storing one or moredata chunks associated with a content file corresponding to tree datastructure 300. “Brick 5” may store one or more data chunks associatedwith a virtual machine content file or one or more data chunks ofmetadata associated with the virtual machine content file. Leaf node 330includes NodeID of “FL5” and a TreeID of “1.” To view the valueassociated with a data key of “5,” tree data structure 300 is traversedfrom root node 302 to intermediate node 314 to leaf node 330.

A file, such as a virtual machine container file, may be comprised of aplurality of data chunks. A brick may store one or more data chunks. Avirtual machine container file is comprised of a plurality of virtualmachine content files and metadata associated with the plurality ofcontent files. Some of the bricks of the file correspond to theplurality of virtual machine content files and some of the bricks of thefile correspond to the metadata associated with the plurality of contentfiles. In the example shown, leaf nodes 322, 324, 326, 328, 330 eachstore a corresponding brick identifier. A metadata store may include adata structure that matches a brick identifier with a correspondinglocation (physical location) of the one or more data chunks comprisingthe brick. In some embodiments, the data structure matches a brickidentifier with a file offset corresponding to metadata and a virtualmachine content file that corresponds to the file offset.

FIG. 3B is a block diagram illustrating an embodiment of adding a filemetadata tree to a tree data structure. In some embodiments, tree datastructure 350 may be created by a storage system, such as secondarystorage system 112. A tree data structure corresponding to a file, suchas a virtual machine container file, is a snapshot tree, but storesmetadata associated with the file (e.g., the metadata associated withthe virtual machine container file). The tree data structurecorresponding to a file can be used to capture different versions of thefile at different moments in time. In some embodiments, the tree datastructure allows a chain of file metadata trees corresponding todifferent versions of a file to be linked together by allowing a node ofa later version of a file metadata tree to reference a node of aprevious version of a file metadata tree. A file metadata tree iscomprised of a root node, one or more levels of one or more intermediatenodes, and one or more leaf nodes.

A root node or an intermediate node of a version of a file metadata treemay reference an intermediate node or a leaf node of a previous versionof a file metadata tree. Similar to the snapshot tree structure, thefile metadata tree structure allows different versions of file data toshare nodes and allows changes to a content file to be tracked. When abackup snapshot is received, a root node of the file metadata tree maybe linked to one or more intermediate nodes associated with a previousfile metadata tree. This may occur when the file is included in bothbackup snapshots.

In the example shown, tree data structure 350 includes a first filemetadata tree comprising root node 302, intermediate nodes 312, 314, andleaf nodes 322, 324, 326, 328, and 330. Tree data structure 350 alsoincludes a second file metadata tree that may be a snapshot of file dataat a particular point in time t+n, for example at time t₁. The secondfile metadata tree is comprised of root node 304, intermediate nodes312, 314, and leaf nodes 322, 324, 326, 328, and 330. The first filemetadata tree may correspond to a first version of a virtual machinecontainer file and the second file metadata tree may correspond to asecond version of the virtual machine container file.

To create a snapshot of the file data at time t+n, a new root node iscreated. The new root node includes the same set of pointers as theoriginal node. In the example shown, root node 304 includes a set ofpointers to intermediate nodes 312, 314, which are intermediate nodesassociated with a previous snapshot. The new root node also includes adifferent NodeID and a different TreeID. The TreeID is the viewidentifier associated with a view of the file metadata tree at aparticular moment in time. In some embodiments, root node 304 isassociated with a current view of the file data. The current view mayrepresent a state of the file data that is up-to-date and is capable ofreceiving one or more modifications to the file metadata tree thatcorrespond to modifications to the file data. The TreeID of a root nodeindicates a snapshot with which the root node is associated. Forexample, root node 302 with a TreeID of “1” is associated with a firstbackup snapshot and root node 304 with a TreeID of “2” is associatedwith a second backup snapshot. In other embodiments, root node 304 isassociated with a snapshot view of the file data. A snapshot view mayrepresent a state of the file data at a particular moment in time in thepast and is not updated.

In the example shown, root node 304 is a copy of root node 302. Similarto root node 302, root node 304 includes the same pointers as root node302. Root node 304 includes a first set of pointers to intermediate node312. The first set of pointers associated with a data key (e.g., “1,”“2,” or “3”) less than or equal the node key indicates that traversing afile metadata tree included in tree data structure 350 from root node304 to intermediate node 312 will lead to a leaf node with a data key of“1,” “2,” or “3.” Root node 304 includes a second set of pointers tointermediate node 314. The second set of pointers associated with a datakey greater than the node key indicates that traversing a file metadatatree included in tree data structure 350 from root node 304 tointermediate node 314 will lead to a leaf node with a data key of “4” or“5.” Root node 304 includes a NodeID of “FR2” and a TreeID of “2.” TheNodeID identifies the name of the node. The TreeID identifies the backupsnapshot with which the node is associated.

FIG. 3C is a block diagram illustrating an embodiment of modifying afile metadata tree of a tree data structure. In the example shown, treedata structure 380 may be modified by a file system manager, such asfile system manager 115. A file metadata tree with root node 304 may bea current view of the file data at time t+n+m, for example, at time t₂.A current view may represent a state of the file data that is up-to-dateand capable of receiving one or more modifications to the file metadatatree that correspond to modifications to the file system data. Because asnapshot represents a perspective of the file data that is “frozen” intime, one or more copies of one or more nodes affected by a change tofile data, are made.

In some embodiments, the file data may be modified such that one of thedata chunks is replaced by another data chunk. When a data chunk of filedata associated with a previous backup snapshot is replaced with a newdata chunk, the data brick storing the data chunk may be different. Aleaf node of a file metadata tree stores a brick identifier associatedwith a particular brick storing the data chunk. To represent thismodification to the file data, a corresponding modification is made to acurrent view of a file metadata tree. The current view of the filemetadata tree is modified because the previous file metadata tree is asnapshot view and can no longer be modified. The data chunk of the filedata that was replaced has a corresponding leaf node in the previousfile metadata tree. A new leaf node in the current view of the filemetadata tree is created, as described herein, that corresponds to thenew data chunk. The new leaf node includes an identifier associated withthe current view. The new leaf node may also store the chunk identifierassociated with the modified data chunk.

In the example shown, a data chunk included in “Brick 4” has beenmodified. The data chunk included in “Brick 4” has been replaced with adata chunk included in “Brick 6.” In some embodiments, the data chunkincluded in “Brick 6” includes a data chunk associated with a virtualmachine content file. In other embodiments, the data chunk included in“Brick 6” includes a data chunk of metadata associated with a virtualmachine content file. At t₂, the file system manager starts at root node304 because that is the root node associated with the file metadata treeat time t₂. The value “Brick 4” is associated with the data key “4.” Thefile system manager traverses tree data structure 380 from root node 304until it reaches a target node, in this example, leaf node 328. The filesystem manager compares the TreeID at each intermediate node and leafnode with the TreeID of the root node. In the event the TreeID of a nodematches the TreeID of the root node, the file system manager proceeds tothe next node. In the event the TreeID of a node does not match theTreeID of the root node, a shadow copy of the node with the non-matchingTreeID is made. For example, to reach a leaf node with a data key of“4,” the file system manager begins at root node 304 and proceeds tointermediate node 314. The file system manager compares the TreeID ofintermediate node 314 with the TreeID of root node 304, determines thatthe TreeID of intermediate node 314 does not match the TreeID of rootnode 304, and creates a copy of intermediate node 314. The intermediatenode copy 316 includes the same set of pointers as intermediate node314, but includes a TreeID of “2” to match the TreeID of root node 304.The file system manager updates a pointer of root node 304 to point tointermediate node 316 instead of pointing to intermediate node 314. Thefile system manager traverses tree data structure 380 from intermediatenode 316 to leaf node 328, determines that the TreeID of leaf node 328does not match the TreeID of root node 304, and creates a copy of leafnode 328. Leaf node 332 is a copy of leaf node 328, but stores the brickidentifier “Brick 6” and includes the same TreeID as root node 304. Thefile system manager updates a pointer of intermediate node 316 to pointto leaf node 332 instead of pointing to leaf node 328.

FIG. 3D is a block diagram illustrating an embodiment of a modified filemetadata tree. The file metadata tree 380 shown in FIG. 3D illustrates aresult of the modifications made to file metadata tree 380 as describedwith respect to FIG. 3C.

FIG. 4 is a flow chart illustrating an embodiment of a process formapping portions of a virtual machine container file to a plurality ofvirtual machine content files and metadata associated with the pluralityof virtual machine content files. In the example shown, process 400 maybe implemented by a storage system, such as secondary storage system112.

At 402, a backup snapshot that includes a virtual machine container fileis received. The backup snapshot is read and determined to include avirtual machine container file. The virtual machine container fileincludes a plurality of virtual machine content files of the virtualmachine and metadata associated with the plurality of virtual machinecontent files. A primary system may perform a backup snapshot of thefile system data according to a backup policy and send the backupsnapshot to a secondary storage system. The virtual machine containerfile corresponds to a particular version of a virtual machine. Some ofthe virtual machine container file is used to store the plurality ofvirtual machine content files and some of the virtual machine containerfile is used to store the metadata associated with the virtual machinecontent files. Portions of the virtual machine container file may beaccessed based on a file offset. When a backup snapshot that includesthe virtual machine container file is received, the portions of thevirtual machine container file that correspond to the plurality ofvirtual machine content files (e.g., file offsets) and the portions ofthe virtual machine container file that correspond to the metadataassociated with the plurality of virtual machine content files isunknown.

At 404, the virtual machine container file is analyzed to determinewhich portions of the virtual machine container file correspond tovirtual machine content files and which portions of the virtual machinecontainer file correspond to metadata associated with the virtualmachine content files, i.e., virtual machine file system metadata. Thevirtual machine container file is comprised of a plurality of datachunks. Some of the data chunks correspond to virtual machine contentfiles and some of the data chunks correspond to metadata associated withthe virtual machine content files. The virtual machine container filemay be read to identify which portions of the virtual machine containerfile correspond to virtual machine content files and which portions ofthe virtual machine container file correspond to metadata associatedwith the virtual machine content files. In some embodiments, the virtualmachine container file is read to identify which portions of the virtualmachine container file correspond to metadata associated with thevirtual machine content files without identifying which portions of thevirtual machine container file correspond to virtual machine contentfiles. The virtual machine container file may include a file table. Thefile table may store the metadata associated with the plurality ofvirtual machine content files. The file offset range of the virtualmachine container file of the file table may be determined.

In some embodiments, the analysis result from determining which portionof the virtual machine content file corresponds to metadata associatedwith the plurality of virtual machine content files is utilized again.For example, the virtual machine file system metadata location/size mayremain constant and be used again in another determination of virtualmachine content file changes. That is, for another version of virtualmachine container file, the portion of the virtual machine containerfile that corresponds to virtual machine file system metadata is knownfrom 404, so the analysis result from 404 may be re-used to determinewhich files of the another version of the virtual machine container filehave changed from a previous version.

In other embodiments, a later version of a virtual machine containerfile is reanalyzed to determine which portion of the virtual machinecontainer file corresponds to the virtual machine file system metadatabecause a location and/or size of the virtual machine file systemmetadata portion may change from version to version.

At 406, the portion of the virtual machine container file correspondingto metadata associated with the plurality of virtual machine contentfiles (the portion corresponding to virtual machine file systemmetadata) is analyzed to determine which file offset range correspondsto which content file metadata. For example, the file table may indicatefile offset ranges of the virtual machine container file that correspondto metadata associated with a virtual machine content file. A virtualmachine container file may be 100 TB and store a plurality of files. Thevirtual machine container file may store a boot sector in a file offsetrange of 0-1 kB region of the virtual machine container file and thefile table in a 1 kB-100 MB region of the virtual machine containerfile. The first entry of the file table may be stored in the file offsetrange of 1 kB-1.1 kB, the second entry of the file table may be storedin the file offset range of 1.1 kB-1.2 kB, and an nth entry of the filetable may be stored in the file offset range of 99.9 MB-100 MB. Thefirst entry may correspond to metadata associated with a first virtualmachine content file, the second entry may correspond to metadataassociated with a second virtual machine content file, and the nth entrymay correspond metadata associated with a nth virtual machine contentfile.

In some embodiments, a data structure is generated and stored. The datastructure may include the virtual machine container file analysisinformation. For example, the data structure may include informationthat indicates a file offset range of the virtual machine container filethat stores metadata associated with the plurality of virtual machinecontent files. The data structure may further associate file offsetranges of metadata associated with a virtual machine content file withits corresponding virtual machine content file. The data structure maybe examined to determine which content files of the virtual machine havechanged in a backup snapshot. For example, a data chunk with a fileoffset in the range 1 kB-1.1 kB may have changed. This file offset rangecorresponds to the metadata associated with a first virtual machinecontent file. Because the metadata associated with the first virtualmachine content file has been modified, the first virtual machinecontent file is determined to have been modified.

FIG. 5 is a flow chart illustrating an embodiment of a process oforganizing file system data of a backup snapshot. In the example shown,process 500 may be implemented by a storage system, such as secondarystorage system 112.

At 502, a first backup snapshot that includes a virtual machinecontainer file corresponding to a first version of a virtual machine isreceived. The first backup snapshot includes file system data receivedfrom a primary system. The file system data includes one or more contentfiles and metadata associated with the one or more content files. Insome embodiments, one of the one or more content files is a virtualmachine container file. The first backup snapshot may be full orincremental backup snapshot of the primary system.

At 504, a tree data structure corresponding to the first backup snapshotis generated. The tree data structure provides a view of the file systemdata corresponding to a backup snapshot. Regardless if the backupsnapshot corresponds to a full or incremental backup snapshot, the treedata structure provides a complete view of the primary system for themoment at which the backup snapshot was performed.

The tree data structure is comprised of a snapshot tree and one or morefile metadata trees. The snapshot tree includes a root node, one or morelevels of one or more intermediate nodes, and one or more leaf nodes.The tree data structure may be traversed from the root node to any ofthe leaf nodes of the snapshot tree. A leaf node of the snapshot treemay include a pointer to a file metadata tree. A file metadata treecorresponds to a content file and stores the metadata associated withthe content file, i.e., the virtual machine file system metadata. Acontent file may be a virtual machine container file. Thus, the filemetadata tree may correspond to a virtual machine container file andstore the metadata associated with the virtual machine container file.The file metadata tree corresponding to a virtual machine container filecorresponds to a version of a virtual machine.

At 506, a second backup snapshot that includes a virtual machinecontainer file corresponding to the second version of the virtualmachine is received. The second backup snapshot includes file systemdata received from a primary system. The file system data includes oneor more content files and metadata associated with the one or morecontent files. In some embodiments, one of the one or more content filesis a virtual machine container file. The second backup snapshot may befull or incremental backup snapshot of the primary system. In the eventthe second backup snapshot corresponds to an incremental backupsnapshot, the second backup snapshot includes the file system data ofthe primary system that was not included in the first backup snapshot.Some of the file system data corresponds to one or more new contentfiles and metadata associated with the one or more new content files.Some of the file system data corresponds to data corresponding to themodified portions of the one or more content files included in aprevious backup snapshot and the associated metadata.

In some embodiments, the file system data includes portions of a virtualmachine container file that were not included in a previous backupsnapshot. The portions of the virtual machine container file may includeone or more new virtual machine content files and associated metadata.The portions of the virtual machine container file may include datacorresponding to the modified portions of the one or more virtualmachine content files included in a previous backup snapshot andmetadata associated with the one or more modified virtual machinecontent files.

At 508, a tree data structure corresponding to the second backupsnapshot is generated. The tree data structure provides a view of thefile system data corresponding to a second backup snapshot. In the eventthe second backup snapshot corresponds to a full backup snapshot, a treedata structure, such as the tree data structure depicted in FIG. 2A maybe generated. In the event the second backup snapshot corresponds to anincremental backup snapshot, a tree data structure corresponding to aprevious backup snapshot may be used as a base tree data structure. Forexample, the tree data structure generated at 504 may be used as a basetree data structure. The base tree data structure may be modified insuch as manner, for example, as depicted in FIGS. 2B-2D to reflect thechanges. Portions of file system data that were added since a previousbackup snapshot may be added to a tree data structure corresponding tothe previous backup snapshot by cloning a root node of the previousbackup snapshot, adding one or more intermediate nodes and one or moreleaf nodes, and updating pointers in a manner as described above.Regardless if the second backup snapshot corresponds to a full orincremental backup snapshot, the tree data structure provides a completeview of the primary system for the moment at which the second backupsnapshot was performed.

In some embodiments, the second backup snapshot includes a secondversion of the virtual machine container file. A file metadata treecorresponding to the virtual machine container file may be updated toreflect the updates. The file metadata tree corresponding to a previousversion of the virtual machine container file may be used as a base treedata structure for the second version of the virtual machine containerfile. For example, the file metadata tree generated at 504 may be usedas a base tree data structure. The new portions of the virtual machinecontainer file may be added to the base tree data structure in a manneras described above with respect to FIGS. 3B-3D.

A leaf node of the snapshot tree that previously pointed to the filemetadata tree corresponding to the previous version of the virtualmachine container file may be updated (e.g., create a copy of the leafnode that points to a root node of the file metadata tree correspondingto the second version of the virtual machine container file) such thatit includes a pointer to a root node of the file metadata treecorresponding to the second version of the virtual machine containerfile.

Using a tree data structure to organize file system data of a backupsnapshot enables version differences between backup snapshots andversion differences between content files (e.g., virtual machinecontainer files) to be easily determined. The differences may bedetermined by traversing the tree data structures and the nodes that arenot shared between the tree data structures correspond to the filesystem data differences between the two backup snapshots. In otherembodiments, the tree data structure associated with the second backupsnapshot and the nodes that are not shared by the two versions may bedetermined based on a view identifier associated with a node. Forexample, a node that has a view identifier associated with the secondversion of the virtual machine container file is not included in thefirst version of the virtual machine container file.

FIG. 6 is a flow chart illustrating an embodiment of a process ofdetermining a modified content file of a virtual machine. In the exampleshown, process 600 may be implemented by a storage system, such assecondary storage system 112.

At 602, the portions of a virtual machine content file that have changedare determined. The portions of a virtual machine content file that havechanged are determined by determining the differences between a firstversion of a virtual machine and a second version of a virtual machineare determined. The differences may be determined by traversing thesnapshot trees corresponding to the first and second versions of thevirtual machine content file and determining the portions of the filemetadata trees corresponding to the virtual machine container file thatare not shared. The differences may be determined by traversing asnapshot tree corresponding to a backup snapshot of file system datathat includes the second version of the virtual machine. The snapshottree corresponding to a backup snapshot of file system data thatincludes the second version of the virtual machine includes a root node,one or more levels of intermediate nodes, and a plurality of leaf nodes.A first leaf node of the plurality of leaf nodes includes a pointer to afile metadata tree corresponding to the first version of the virtualmachine and a second leaf node of the plurality of leaf nodes includes apointer to a file metadata tree corresponding to the second version ofthe virtual machine. The snapshot tree may be traversed from the rootnode to the first leaf node and to the second leaf node. The pointersincluded in the first and second leaf nodes may be followed to the filemetadata trees corresponding to the first and second versions of thevirtual machine.

A file metadata tree includes a root node, one or more levels of one ormore intermediate nodes associated with the root node, and one or moreleaf nodes associated with an intermediate node of the lowestintermediate level. A file metadata tree is similar to a snapshot tree,but a leaf node of a file metadata tree includes an identifier of a databrick storing one or more data chunks of the file or a pointer to thedata brick storing one or more data chunks of the file. The filemetadata tree corresponding to the first version of the virtual machineand the file metadata tree corresponding to the second version of thevirtual machine share one or more leaf nodes and do not share one ormore leaf nodes. In some embodiments, at least one of the leaf nodesthat is not shared between the file metadata tree corresponding to thefirst version of the virtual machine and the file metadata treecorresponding to the second version of the virtual machine is includedin the file metadata tree corresponding to the first version of thevirtual machine, but is not included in the file metadata treecorresponding to the second version of the virtual machine. In otherembodiments, at least one of the leaf nodes that is not shared betweenthe file metadata tree corresponding to the first version of the virtualmachine and the file metadata tree corresponding to the second versionof the virtual machine is included in the file metadata treecorresponding to the second version of the virtual machine, but is notincluded in the file metadata tree corresponding to the first version ofthe virtual machine. The one or more leaf nodes that are included in thefile metadata tree corresponding to the second version of the virtualmachine, but are not included in the file metadata tree corresponding tothe first version of the virtual machine are analyzed to determine oneor more data bricks associated with the second version of the virtualmachine that are not included in the first version of the virtualmachine.

At 604, it is determined whether a changed portion of the virtualmachine container file corresponds to a metadata portion of the virtualmachine container file, i.e., the virtual machine file system metadata.A data brick of the one or more determined data bricks may correspond toa virtual machine content file or metadata associated with the virtualmachine content file. A data brick has an associated file offset withinthe virtual machine container file. A data structure storing suchinformation may be examined to determine whether the file offset of thedata brick corresponds to a portion of the virtual machine containerfile storing virtual machine content files or a portion of the virtualmachine container file storing metadata associated with the virtualmachine content files. A data brick corresponds to the portion of thevirtual machine container file storing metadata associated with thevirtual machine content files in the event the data brick has a fileoffset that is within a file offset range associated with the metadataassociated with the plurality of virtual machine content files (e.g.,the data brick has a file offset associated with a master file table ofthe virtual machine container file). Whether a changed portion of thevirtual machine container file corresponds to a metadata portion of thevirtual machine container file may be determined by intersecting a fileoffset associated with a data brick with a file offset range associatedwith the metadata associated with the plurality of virtual machinecontent files. A changed portion corresponds to metadata associated withthe plurality of virtual machine content files in the event the fileoffset associated with the data brick is within the file offset rangeassociated with the metadata associated with the plurality of virtualmachine content files.

At 606, a changed portion of the virtual machine container file is readto determine which virtual machine content file was modified. In someembodiments, the contents of the changed portion indicates that avirtual machine content file was modified. For example, the changedportion may correspond to metadata associated with a virtual machinecontent file. The metadata may store filename of a virtual machinecontent file and a timestamp that indicates that the virtual machinecontent file with which the metadata is associated, has changed. Forexample, the metadata may store a timestamp that indicates the virtualmachine content file was modified after a last backup snapshot. Themetadata associated with a virtual machine content file may be read andthe virtual machine content file with which the metadata is associated,is determined to have changed.

In other embodiments, a virtual machine content file is determined tohave been modified based on a file offset associated with one of theanalyzed leaf nodes. The secondary storage system may manage a datastructure (e.g., map) that associates file offset ranges with virtualmachine content files. A leaf node may store a brick identifier or apointer to a brick storing one or more data chunks associated with thevirtual machine container file. A brick has a corresponding file offsetand may be used to determine that the brick corresponds to metadataassociated with a virtual machine content file. The file offsetcorresponding to the brick may compared to the file offset rangeassociated with metadata associated with the plurality of virtualmachine content files. In the event the file offset corresponding to thebrick is included in a file offset range associated with metadataassociated with a virtual machine content file, the virtual machinecontent file corresponding to the file offset range may be determined tohave been modified or added between virtual machine versions. Forexample, a data brick with a file offset of 1.1 kB-1.2 kB corresponds tothe metadata associated with a first virtual machine content file andindicates that the first virtual machine content file has been modifiedor added, a data brick with a file offset of 1.1 kB-1.2 kB correspondsto the metadata associated with a second virtual machine content fileand indicates that second virtual machine content file has been modifiedor added, and a data brick with a file offset of 99.9 MB-100 MBcorresponds to the metadata associated with an nth virtual machinecontent file and indicates that the nth virtual machine content file hasbeen modified or added.

One or more virtual machine content files that have changed since aprevious virtual machine backup may be quickly identified byintersecting the data bricks identified by traversing the snapshot treewith the portion of a master file table corresponding to modified filesbecause the files in the master file table are small (e.g., 1 kB). Theamount of time needed to read a file in the master file table pales incomparison to the amount of time needed to read all of the virtualmachine metadata. The amount of time needed to read a subset of themaster file table is proportional to the number of virtual machinecontent files that have changed since a last backup. For example, a 100TB virtual machine container file may have 100 GB of metadata. Eachvirtual machine content file may have a corresponding metadata file inthe master file table that is 1 kB in size. Traversing the snapshottrees may identify 10 files have changed since a last backup. Thestorage system may read 10 kB in data (10 files, each metadata file is 1kB) to determine the one or more virtual machine content files that havechanged since a pervious virtual machine backup instead of reading the100 GB of metadata.

Determining which virtual machine content files are not shared betweenvirtual machine versions using the techniques disclosed herein hasseveral advantages. First, an index may be created that lists the one ormore virtual machine content files associated with a virtual machineversion. The amount of time needed to create the index is reducedbecause the one or more virtual machine content files that have beenmodified or added since a previous virtual machine version may bequickly identified using the tree data structure disclosed herein. Theindex associated with a previous version of the virtual machine may bequickly updated to include the one or more identified virtual machinecontent files. Second, a version of a virtual machine content fileincluded within a virtual machine version may be determined. This mayenable a user to recover a particular version of a virtual machinecontent file. Because a virtual machine container file includes aplurality of virtual machine content files, it is difficult and timeconsuming to determine the virtual machine content files included in thevirtual machine container file and whether any of the virtual machinecontent files has changed since a previous version of the virtualmachine container file. Third, a virus scan of a virtual machine may beperformed a lot faster. For example, a first version of a virtualmachine, i.e., the entire virtual machine content file may have beenscanned. A second version of a virtual machine may also be scanned, butinstead of scanning the entire contents of the second version of thevirtual machine, the one or more virtual machine content files that havebeen modified or added since the first version of the virtual machinemay be scanned. The one or more virtual machine content files may beidentified using the techniques disclosed herein. Subsequently, a virusscanner may be applied to the portions of the virtual machine containerfile corresponding to the one or more identified virtual machine contentfiles. Given the size of a virtual machine container file (e.g., 100TB), the techniques disclosed herein significantly reduce the amount oftime to perform a virus scan of the virtual machine container file.Fourth, the virtual machine container file may be analyzed to determinehow much data has changed between virtual machine versions and whichportions of the virtual machine container file have changed. This mayallow a user of the virtual machine container file to determine whichportions of the virtual machine are frequently used and/or critical tothe operation of the virtual machine.

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

What is claimed is:
 1. A method, comprising: generating a map thatassociates a virtual machine container file offset range of metadatawith a corresponding virtual machine content file included in a virtualmachine container file; determining one or more differences between afirst version of the virtual machine container file and a second versionof the virtual machine container file based on traversing a tree datastructure corresponding to the first version of the virtual machinecontainer file and a tree data structure corresponding to the secondversion of the virtual machine container file; determining, based on oneor more file offset ranges of metadata in the map corresponding to theone or more determined differences between the first version of thevirtual machine container file and the second version of the virtualmachine container file, one or more virtual machine content files in thesecond version of the virtual machine container file that have changedsince the first version of the virtual machine container file, includingby reading, at the one or more file offset ranges of metadata in the mapcorresponding to the one or more determined differences between thefirst version of the virtual machine container file and the secondversion of the virtual machine container file, metadata associated withthe one or more virtual machine content files in the second version ofthe virtual machine container file that have changed since the firstversion of the virtual machine container file; and scanning the one ormore virtual machine content files in the second version of the virtualmachine content file that have changed since the first version of thevirtual machine container file.
 2. The method of claim 1, whereingenerating the map includes analyzing the virtual machine container fileto determine a location of a file table.
 3. The method of claim 2,wherein the file table stores corresponding file offset ranges of thevirtual machine container file for metadata associated with each virtualmachine content file included in the virtual machine container file. 4.The method of claim 1, wherein traversing the tree data structurecorresponding to the first version of the virtual machine container fileand the tree data structure corresponding to the second version of thevirtual machine container file determines one or more nodes that are notshared between the tree data structures.
 5. The method of claim 4,further comprising determining that at least one of the one or moredetermined nodes is associated with the second version of the virtualmachine container file.
 6. The method of claim 5, wherein the at leastone of the one or more determined nodes is associated with acorresponding file offset range of the second version of the virtualmachine container file.
 7. The method of claim 6, wherein thecorresponding file offset range of the second version of the virtualmachine container file corresponds to metadata associated with a firstvirtual machine content file included in the second version of thevirtual machine container file.
 8. The method of claim 7, furthercomprising reading the metadata associated with the first virtualmachine content file in the corresponding file offset range of thesecond version of the virtual machine container file to determine thatthe first virtual machine content file corresponding the read fileoffset range has changed.
 9. The method of claim 8, wherein the metadataassociated with the first virtual machine content file includes at leasta filename and a timestamp.
 10. The method of claim 1, furthercomprising generating an index that identifies the one or more virtualmachine content files included in the second version of the virtualmachine container file.
 11. The method of claim 1, wherein traversingthe tree data structure corresponding to the first version of thevirtual machine container file and the tree data structure correspondingto the second version of the virtual machine container file determinesan amount of data that has changed between the first version of thevirtual machine container file and the second version of the virtualmachine container file.
 12. A computer program product, the computerprogram product being embodied in a non-transitory computer readablestorage medium and comprising computer instructions for: generating amap that associates a virtual machine container file offset range ofmetadata with a corresponding virtual machine content file included in avirtual machine container file; determining one or more differencesbetween a first version of the virtual machine container file and asecond version of the virtual machine container file based on traversinga tree data structure corresponding to the first version of the virtualmachine container file and a tree data structure corresponding to thesecond version of the virtual machine container file; determining, basedon one or more file offset ranges of metadata in the map correspondingto the one or more determined differences between the first version ofthe virtual machine container file and the second version of the virtualmachine container file, one or more virtual machine content files in thesecond version of the virtual machine container file that have changedsince the first version of the virtual machine container file, includingby reading, at the one or more file offset ranges of metadata in the mapcorresponding to the one or more determined differences between thefirst version of the virtual machine container file and the secondversion of the virtual machine container file, metadata associated withthe one or more virtual machine content files in the second version ofthe virtual machine container file that have changed since the firstversion of the virtual machine container file; and scanning the one ormore virtual machine content files.
 13. The computer program product ofclaim 12, wherein traversing the tree data structure corresponding tothe first version of the virtual machine container file and the treedata structure corresponding to the second version of the virtualmachine container file determines one or more nodes that are not sharedbetween the tree data structures.
 14. The computer program product ofclaim 13, further comprising computer instructions for determining thatat least one of the one or more determined nodes is associated with thesecond version of the virtual machine container file.
 15. The computerprogram product of claim 14, wherein the at least one of the one or moredetermined nodes is associated with a corresponding file offset range ofthe second version of the virtual machine container file.
 16. Thecomputer program product of claim 15, wherein the corresponding fileoffset range of the second version of the virtual machine container filecorresponds to metadata associated with a first virtual machine contentfile included in the second version of the virtual machine containerfile.
 17. The computer program product of claim 16, further comprisingcomputer instructions for reading the metadata associated with the firstvirtual machine content file in the corresponding file offset range ofthe second version of the virtual machine container file to determinethat the first virtual machine content file corresponding the read fileoffset range has changed.
 18. The computer program product of claim 17,wherein the metadata associated with the first virtual machine contentfile includes at least a filename and a timestamp.
 19. The computerprogram product of claim 12, further comprising computer instructionsfor generating an index that identifies the one or more virtual machinecontent files included in the second version of the virtual machinecontent file.
 20. A system, comprising: a processor configured to:generate a map that associates a virtual machine container file offsetrange of metadata with a corresponding virtual machine content fileincluded in a virtual machine container file; determine one or moredifferences between a first version of the virtual machine containerfile and a second version of the virtual machine container file based ontraversing a tree data structure corresponding to the first version ofthe virtual machine container file and a tree data structurecorresponding to the second version of the virtual machine containerfile; determine, based on one or more file offset ranges of metadata inthe map corresponding to the one or more determined differences betweenthe first version of the virtual machine container file and the secondversion of the virtual machine container file, one or more virtualmachine content files in the second version of the virtual machinecontainer file that have changed since the first version of the virtualmachine container file, including by reading, at the one or more fileoffset ranges of metadata in the map corresponding to the one or moredetermined differences between the first version of the virtual machinecontainer file and the second version of the virtual machine containerfile, metadata associated with the one or more virtual machine contentfiles in the second version of the virtual machine container file thathave changed since the first version of the virtual machine containerfile; and scan the one or more virtual machine content files; and amemory coupled to the processor and configured to provide the processorwith instructions.